ChildWIN: Child Welfare Workforce Innovation

The Children’s Home Society of Florida (CHS) embarked on an effort to improve child outcomes and workforce outcomes through an initiative called ChildWIN. Consisting of three components (career ladder, reduced caseloads, and Solution-Based Casework training), this initiative was fully implemented in Seminole County, partially implemented in Orange County, and not implemented in the Treasure Coast. To evaluate this initiative, the School of Social Work at the University of Central Florida conducted an analysis of child outcomes across all three geographical regions, an analysis of caseworker turnover and caseworker job satisfaction across all three geographical regions, and an analysis of focus group discussions among caseworkers in Seminole County. This report contains the results of all segments of the evaluation. Results are provided in four forms: a comparison of each region’s performance to the CHS standards, a comparison across regions to identify the effects of ChildWIN, a summary of caseworker perceptions on the effects of ChildWIN, and a consideration of additional variables found to correlate with the outcomes. May 4, 2017 Principal Investigator: Julie Steen, PhD, MSW, Associate Professor University of Central Florida School of Social Work Co-Principal Investigator: Reshawna Chapple, PhD, LCSW, Assistant Professor University of Central Florida School of Social Work Key Project Staff: Tara Hormell, MA, Executive Director Children’s Home Society Central Florida and Brevard Divisions Andry Sweet, MS, Chief of Staff Children’s Home Society Funded through a contract with the Florida Institute for Child Welfare CONTENTS Abstract ..........................................................................................................................................1 Executive Summary of Findings ...................................................................................................2 ChildWIN Evaluation .....................................................................................................................3 ChildWIN Initiative .......................................................................................................... 4 Methodology................................................................................................................... 4 Results ...........................................................................................................................................5 1: Child Safety ................................................................................................................ 5 2: Child Permanency ...................................................................................................... 7 3: Workforce Stability ................................................................................................... 12 Discussion ....................................................................................................................................17 Policy Recommendations ...........................................................................................................18 RESEARCH REPORT Executive Summary of Findings This executive summary provides a brief overview of the results, which are explained in more detail in the body of the report. Results are available for each of the major categories of outcomes (safety, permanency, and workforce stability). Readers can find all statistical analyses and qualitative data in this report. Safety Safety was one of the three key domains of outcomes included in this study. Results were mixed with the qualitative data reflecting a positive impact and the quantitative data providing no support for an effect. In focus group discussions, caseworkers reported that reduced caseloads in particular were successful in increasing safety by providing them with extra time to investigate their suspicions of abuse and neglect. Note that this effect may result in improved identification of maltreatment, which may in turn increase the reabuse rate. This issue may explain why the quantitative data found no effect. Statistics regarding reabuse may be an indicator of the quality of investigation and identification rather than an indicator of safety; therefore, higher reabuse rates may not necessarily represent higher rates of maltreatment. The quantitative results focused on two dimensions of safety (reabuse during in-home supervision and reabuse during out-of-home care). Safety During In-Home Supervision • While Seminole County and the Treasure Coast met the CHS standard for safety during in-home supervision, Orange County did not meet this standard. • The quantitative data provide little evidence of ChildWIN’s effect on safety during in-home supervision, since the control group (Treasure Coast) performed at the same level as the county implementing the full ChildWIN model (Seminole County) and outperformed the county implementing the Solution-Based Casework training (Orange County). • Two variables were found to be significantly associated with safety during in-home supervision. White children and children served by caseworkers with high caseloads were more likely to be reabused during inhome supervision. Safety During Out-of-Home Care • All three geographical regions met the CHS standard for safety during out-of-home care. • The quantitative data provide little evidence of ChildWIN’s effect on safety during out-of-home care, since the control group (Treasure Coast) outperformed the other two regions. • Four variables were found to be significantly associated with safety during out-of-home care. Black children and older children were more likely to be reabused during out-of-home care. The caseload level and caseworker turnover were also associated with this outcome, with higher caseload levels and turnover being found alongside higher reabuse rates. Permanency Permanency was the second key outcome included in the study. Results in this domain were also mixed with the qualitative data reflecting a positive impact and the quantitative data providing only partial evidence of a positive impact. Caseworkers reported that the reduced caseload allowed them to invest more time in building rapport with families and completing tasks that advanced the cases forward. They believed that this would improve permanency. Caseworkers also believed that the Solution-Based Casework training had the potential to improve permanency, though they mentioned a number of barriers that inhibited the full implementation of the model. The quantitative data provided little evidence of an effect in the areas of permanency following out-of-home care and placement stability during out-of-home care. On the other hand, partial support for a positive effect on permanency following in-home supervision was demonstrated through the relative success seen in Orange County, which received the Solution-Based Casework training. Permanency Following In-Home Supervision • None of the geographical regions met the CHS standard for permanency of children receiving in-home supervision. Note that this result may be due to the short duration of the study (seven months). • The quantitative data provide some evidence of Solution-Based Casework’s positive effect on Orange County’s ability to achieve permanency for children receiving in-home supervision. On the other hand, Seminole County, which also received Solution-Based Casework training, had the lowest percentage of children achieving permanency. • No variables were found to be significantly associated with permanency following in-home supervision. Permanency Following Out-of-Home Care • None of the geographical regions met the CHS standard for permanency of children in out-of-home care. Note that this result may be due to the short duration of the study (seven months). • The quantitative data provide little evidence of ChildWIN’s effect on permanency for children in out-of-home care, since the control group (Treasure Coast) outperformed the other two regions. • Two variables were significantly associated with successful transition to permanency following out-of-home care. Black children and older children were less likely to transition to permanency. Placement Stability During Out-of-Home Care • All of the geographical regions met the CHS standard for stability during out-of-home placement. • The quantitative data provide little evidence of ChildWIN’s effect on out-of-home placement stability, since the control group (Treasure Coast) outperformed the other two regions. • Two variables were found to significantly correlate with placement stability. Black children and older children were significantly less likely to experience placement stability. FLORIDA INSTITUTE FOR CHILD WELFARE 2 Workforce Stability Workforce stability was the third key outcome included in the study. The majority of the evidence, both quantitative and qualitative, provide support for a positive impact on workforce stability. In focus group discussions, caseworkers described multiple positive impacts. They said that the Solution-Based Casework training reignited the aspirations they had when they initially took the job, the career ladder provided them with an incentive to remain in the position, and the reduced caseloads allowed them to have more success with their clients and invest in their own home life. These impacts are supported by the job satisfaction scores of Seminole County’s caseworkers. The turnover rates also provide some support for a positive impact, with Seminole and Orange Counties outperforming the control group. Caseworker Turnover • All of the geographical regions met the CHS standard for caseworker turnover. • The quantitative data provide some evidence of ChildWIN’s effect on turnover, since the county implementing the reduced caseload and career ladder (Seminole) outperformed the control group (Treasure Coast). However, Orange County, which did not implement the reduced caseload or the career ladder, had the lowest turnover rate. • No significant relationship was found between turnover and caseload level. Caseworker Job Satisfaction • All of the geographical regions met the CHS standard for caseworkers’ satisfaction with their work and their co-workers at both Time 1 and Time 2. None of the regions met the standard for caseworkers’ satisfaction with their pay or promotion opportunities at Time 1 or Time 2. Orange and Seminole Counties met the standard for satisfaction with supervision at Time 1, while all three regions met this standard at Time 2. A similar pattern is seen for the Job In General satisfaction score with only Orange and Seminole Counties meeting the standard at Time 1 and all regions meeting the standard at Time 2. • The quantitative data indicate that ChildWIN had a positive impact on the job satisfaction scores of Seminole County caseworkers, though most of this positive impact was short-term in nature. • In a few circumstances, caseload levels were found to correlate with job satisfaction scale scores. Higher caseload levels were found alongside lower satisfaction with pay, promotion opportunities, and the job in general. ChildWIN Evaluation Mission, Goals, and Objectives The Children’s Home Society of Florida (CHS) is directed towards the mission of “building bridges to success for children.” In the child welfare context, this mission is accomplished through the promotion of child safety and child permanency. Further, the mission is supported through the promotion of child welfare workforce stability. In line with this mission, the Children’s Home Society of Florida selected six objectives to guide their efforts in 2016. The first two objectives focus on child safety and read as follows: 1. 95 percent of the children receiving in-home services at the time of pretest will not be abused or neglected by the time of posttest. 2. 95 percent of the children in out-of-home care at the time of pretest will not be abused or neglected by the time of posttest. Child permanency is another important goal that is represented by two of the six objectives. 1. 50 percent of the children receiving in-home services or out-of-home care at the time of pretest will have a permanent placement by the time of posttest. 2. 50 percent of the children in out-of-home care at the time of posttest will have fewer than three placements by the time of posttest. The third goal focuses on child welfare workforce stability. The remaining two objectives align with this goal: 1. 70 percent of the caseworkers employed at the time of pretest will remain in their positions by the time of posttest. 2. 70 percent of the caseworkers employed at the time of posttest will express high levels of job satisfaction as indicated by subscale scores in the high end of the range of possible scores. These six objectives served as a guidepost for the ChildWIN initiative and the ChildWIN evaluation. FLORIDA INSTITUTE FOR CHILD WELFARE 3